Accessing multiple user states concurrently in a configurable IC

ABSTRACT

Some embodiments of the invention provide a configuration/debug network for configuring and debugging a configurable integrated circuit (IC). The configurable IC in some embodiments includes configurable resources (e.g., configurable logic resources, routing resources, memory resources, etc.) that can be grouped in conceptual configurable tiles that are arranged in several rows and columns. Some embodiments allow tiles to be individually addressed, globally addressed (i.e., all addressed together), or addressed based on their tile types. The configurable IC includes numerous user-design state elements (“UDS elements”) in some embodiments. In some embodiments, the configuration/debug network has a streaming mode that can direct various circuits in one or more configurable tiles to stream out their data during the operation of the configurable IC. Accordingly, in the embodiments where the configuration/debug network connects to some or all of the UDS elements, the configurable/debug network can be used in a streaming mode to stream out data from the UDS elements of the tiles, in order to identify any errors in the operation of the IC. In other words, the streaming of the data from the UDS elements can be used to debug the operation of the configurable IC. In some embodiments, the configuration/debug network has a broadcasting mode that can direct various resources (e.g., memories, storage elements, etc.) in one or more configurable tiles to store the same data. For instance, the broadcasting mode can be used to initialize the memory blocks in the configurable memory tiles.

CLAIM OF BENEFIT TO PRIOR APPLICATIONS

This Application is a continuation application of U.S. patentapplication Ser. No. 11/375,562, entitled “Accessing Multiple UserStates Concurrently in a Configurable IC,” filed Mar. 13, 2006 now U.S.Pat. No. 7,788,478 which is incorporated herein by reference. U.S.patent application Ser. No. 11/375,562 claims priority to prior filedU.S. Provisional Patent Application 60/699,463, entitled “Configurationand Debug Network for Configurable ICs,” filed Jul. 15, 2005.

CROSS REFERENCE TO RELATED APPLICATIONS

This Application is related to the following applications: U.S. patentapplication Ser. No. 11/375,362, filed Mar. 13, 2006, now issued as U.S.Pat. No. 7,375,550; U.S. patent application Ser. No. 12/050,897, filedMar. 18, 2008, now issued as U.S. Pat. No. 7,548,090; U.S. patentapplication Ser. No. 11/375,363, filed Mar. 13, 2006, now issued as U.S.Pat. No. 7,443,196; U.S. patent application Ser. No. 12/235,581, filedSep. 22, 2008, now issued as U.S. Pat. No. 7,728,617; U.S. patentapplication Ser. No. 12/754,603, filed Apr. 5, 2010, now issued as U.S.Pat. No. 8,115,510; U.S. patent application Ser. No. 11/375,370, filedMar. 13, 2006, now issued as U.S. Pat. No. 7,512,850; U.S. patentapplication Ser. No. 11/375,369, filed Mar. 13, 2006, now issued as U.S.Pat. No. 7,550,991; U.S. patent application Ser. No. 11/375,364, filedMar. 13, 2006, now issued as U.S. Pat. No. 7,492,186; U.S. patentapplication Ser. No. 12/106,257, filed Apr. 18, 2008, now issued as U.S.Pat. No. 7,696,780; U.S. patent application Ser. No. 12/754,604, filedApr. 5, 2010, now issued as U.S. Pat. No. 8,067,960; U.S. patentapplication Ser. No. 13/281,425, filed Oct. 25, 2011, now publised asU.S. Patent Publication 2012-0098567; and U.S. patent application Ser.No. 11/375,561, filed Mar. 13, 2006, now issued as U.S. Pat. No.7,548,085.

FIELD OF THE INVENTION

The present invention is directed towards accessing multiple user statesconcurrently in a configurable IC.

BACKGROUND OF THE INVENTION

The use of configurable integrated circuits (“IC's”) has dramaticallyincreased in recent years. One example of a configurable IC is a fieldprogrammable gate array (“FPGA”). An FPGA is a field programmable ICthat usually has configurable logic and interconnect circuits that aresurrounded by input/output (i/o) circuits.

The configurable logic circuits (also called logic blocks) are typicallyarranged as an internal array of circuits. A configurable logic circuitcan be configured to perform a number of different functions. Aconfigurable logic circuit typically receives a set of input data and aset of configuration data that is often stored close to the logiccircuit. From the set of functions that the logic circuit can perform,the configuration data set specifies a particular function that thiscircuit is to perform on the input data set. Such a logic circuit issaid to be configurable, as the configuration data set “configures” thelogic circuit to perform a particular function.

These logic circuits are connected together through numerousconfigurable interconnect circuits (also called interconnects). Aconfigurable interconnect circuit connects a set of input data to a setof output data based on a set of configuration data that it receives.The configuration bits specify how the interconnect circuit shouldconnect the input data set to the output data set. The interconnectcircuit is said to be configurable, as the configuration data set“configures” the interconnect circuit to use a particular connectionscheme that connects the input data set to the output data set in adesired manner. In some FPGA's, the configuration data set of aconfigurable logic or interconnect set can be modified by writing newdata in SRAM cells that store the configuration data set.

Most configurable IC's need to load configuration data in storage cellsthat store such data for use by configurable logic and/or interconnectcircuits. Prior mechanisms for loading such data are a bit slow as theyprovide only low bandwidth architectures for loading configuration data.In addition, these prior mechanisms do not provide a random access wayfor loading the configuration data.

Therefore, there is a need in the art for a better mechanism for loadingconfiguration data in storage cells used for storing configuration datafor configurable logic and/or interconnect circuits. There is also aneed for a better ways to monitor and debug operations of configurableIC's. Ideally, the mechanism for loading configuration data could alsobe partly used to monitor and debug the configurable IC.

SUMMARY OF THE INVENTION

Some embodiments of the invention provide a configuration/debug networkfor configuring and debugging a configurable integrated circuit (IC).The configurable IC in some embodiments includes configurable resources(e.g., configurable logic resources, routing resources, memoryresources, etc.) that can be grouped in conceptual configurable tilesthat are arranged in several rows and columns. In some embodiments, eachconfigurable tile receives a set of lines that are part of theconfiguration/debug network.

Some embodiments use a packet switching technology to route data to andfrom the resources in the configurable tiles through theconfiguration/debug network. Over the lines of the configuration/debugnetwork, some embodiments can route variable length data packets to eachconfigurable tile in a sequential or random access manner.

In some embodiments, each packet includes several frames, with aninitial set of frames in a packet specifying the routing of the packetto a configurable tile. For instance, the first two frames of a packetmight respectively identify the column and then the row of theconfigurable tile to be configured.

Some embodiments have different types of configurable tiles, such as oneor more configurable logic tile types, one or more configurable memorytile types, one or more configurable edge tiles, etc. Some of theseembodiments allow tiles to be individually addressed, globally addressed(i.e., all addressed together), or addressed based on their tile types.

The configurable IC includes numerous user-design state elements (“UDSelements”) in some embodiments. UDS elements are elements that storevalues that at any particular time define the overall user-design stateof the configurable IC at that particular time. Examples of suchelements include latches, registers, memories, etc. The configurable ICof some embodiments might not include all such forms of UDS elements, ormight include other types of UDS elements.

In some embodiments, the configuration/debug network connects to some orall of the UDS elements (e.g., latches, registers, memories, etc.) ofthe configurable IC. In some embodiments, the configuration/debugnetwork has a streaming mode that can direct various circuits in one ormore configurable tiles to stream out their data during the operation ofthe configurable IC. Accordingly, in the embodiments where theconfiguration/debug network connects to some or all of the UDS elements,the configurable/debug network can be used in a streaming mode to streamout data from the UDS elements of the tiles, in order to identify anyerrors in the operation of the IC. In other words, the streaming of thedata from the UDS elements can be used to debug the operation of theconfigurable IC.

The streaming mode is used in some embodiments to form a logic analyzer,which may be on or off the same IC die that includes the configurabletiles. For instance, some embodiments include a trace buffer on the sameIC die as the configurable tiles. This trace buffer can then be used torecord the data that is output form one or more tiles during thestreaming mode operation of the configurable IC. In other words, thetrace buffer can be used to implement an “on-chip” logic analyzer inconjunction with the streaming mode operation of the IC. An “off-chip”logic analyzer can also be formed by using an off-chip trace buffer(i.e., a buffer that is not on the same die as the configurable IC)while using the streaming mode operation of the IC's configuration/debugnetwork.

Some embodiments also use the configuration/debug network to performcheckpointing operations. Checkpointing is a sub-operation of a debugoperation. The checkpointing process of some embodiments periodicallystops the configurable IC's operations (e.g., stops the IC's operationsevery few million cycles). At each stoppage of the IC's operations, thecheckpointing process uses the configuration/debug network to retrievethe configurable IC's state at that time (e.g., to retrieve the valuestored by each UDS element of the configurable IC at that time). Oncethis process has retrieved the configurable IC's state, it causes theconfigurable IC to resume its operations. When an error is detectedduring the debug operation (i.e., after a “crash”), a user or debuggingapplication then loads the mostly recently checkpointed IC state (i.e.,stored IC state) within the IC, and resumes the debug operation in amore deliberate manner (e.g., slower or under more supervision) in orderto identify the cause of the error.

In some embodiments, the configuration/debug network has a broadcastingmode that can direct various resources (e.g., memories, storageelements, etc.) in one or more configurable tiles to store the samedata. For instance, the broadcasting mode can be used to initialize thememory blocks in the configurable memory tiles.

The configuration/debug network of some embodiments is a pipelinednetwork that can carry multiple instructions and data sets for multipletiles concurrently. This pipelined nature of the network allows thenetwork to rapidly configure the IC. This rapid operation, in turn,allows the configurable IC to re-load configuration data for a first setof configurable circuits while a second set of configurable circuits areoperating (i.e., while the IC is operating).

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the invention are set forth in the appendedclaims. However, for purpose of explanation, several embodiments of theinvention are set forth in the following figures.

FIG. 1 illustrates an example of a configurable IC that includesnumerous configurable tiles.

FIG. 2 illustrates an example of a data packet.

FIGS. 3, 4, and 5 illustrate an example of how an initial set of framesin a packet might specify the routing of a packet to a configurabletile.

FIG. 6 illustrates the configurable circuit architecture of someembodiments of the invention.

FIG. 7 provides one possible physical architecture of the configurableIC illustrated in FIG. 6.

FIG. 8 illustrates two examples of configurable memory tiles.

FIG. 9 illustrates a configuration/debug controller that formulatesconfiguration/debug packets that are used to specify the operation ofthe configurable tiles of a configurable IC during configuration/debugoperations.

FIG. 10 illustrates an example of a data packet that can be routed toone or more configurable tile sets.

FIG. 11 illustrates examples of some opcodes that are used in someembodiments.

FIGS. 12 and 13 illustrate two examples of packets used to routeconfiguration/debug information over the configuration/debug network ofsome embodiments.

FIG. 14 provides an overview of the configuration and debug network ofsome embodiments.

FIG. 15 illustrates the circuits of the configuration/debug network in atop tile.

FIG. 16 illustrates how some embodiments write a value into a latchstorage element.

FIG. 17 illustrate the decoder logic of a conceptual memory block forsome embodiments of the invention.

FIG. 18 conceptually illustrates the masking logic of the top tile ofsome embodiments of the invention.

FIG. 19 illustrates the network structure of a center tile in someembodiments of the invention.

FIG. 20 illustrates the network structure of a bottom tile in someembodiments of the invention.

FIG. 21 illustrates a process that the configuration controller canperform to operate the configuration/debug network in a broadcastingmode.

FIG. 22 illustrates a process that the configuration controller canperform to operate the configuration/debug network in a streaming mode.

FIG. 23 illustrates an example of a trace buffer according to someembodiments of the invention.

FIG. 24 conceptually illustrates the logic analyzer functionality ofsome embodiments of the invention.

FIG. 25 illustrates a trace buffer according to some embodiments of theinvention.

FIG. 26 illustrates a debugging process that conceptually illustrates aseries of debugging operations some of which use the configuration/debugnetwork of some embodiments to perform checkpointing.

FIG. 27 illustrates a debugger that directs the debugging process ofFIG. 26.

FIG. 28 illustrates an example of the fast swapping of configurationsduring the user-design operation of an IC.

DETAILED DESCRIPTION

In the following description, numerous details are set forth for purposeof explanation. However, one of ordinary skill in the art will realizethat the invention may be practiced without the use of these specificdetails. In other instances, well-known structures and devices are shownin block diagram form in order not to obscure the description of theinvention with unnecessary detail.

I. Overview

Some embodiments of the invention provide a configuration/debug networkfor configuring and debugging a configurable integrated circuit (IC).The configurable IC in some embodiments includes configurable resources(e.g., configurable logic resources, routing resources, memoryresources, etc.) that can be grouped in conceptual configurable tilesthat are arranged in several rows and columns.

FIG. 1 illustrates an example of a configurable IC 100 that includesnumerous configurable tiles 105. As shown in this figure, eachconfigurable tile 105 receives a set of lines 110 that are part of theconfiguration/debug network. Some embodiments use a packet switchingtechnology to route data to and from the resources in the configurabletiles. Hence, over the lines 110, these embodiments can route variablelength data packets to each configurable tile in a sequential or randomaccess manner.

FIG. 2 illustrates an example of a data packet 200. As shown in thisfigure, the data packet 200 includes several data frames 205. In someembodiments, an initial set of frames (e.g., first one or two frames) ofthe packet identifies configurable tiles for routing the remainingframes of the data packet. These remaining frames can then containconfiguration and/or debug data for configuring the tile or performingdebug operations on the tile.

FIGS. 3, 4, and 5 illustrate an example of how an initial set of framesin a packet might specify the routing of a packet to a configurable tile315. In this example, the first two frames 305 and 310 of the packet 300respectively identify the column and then the row of the configurabletile 315 to be configured. As shown in FIG. 4, the column-identifyingframe 305 is used by a column selector at the top of the configurabletile array 325 to route a packet down the column of the addressedconfigurable tile 310. The tile-identifying frame 310 then allows a tileselector in the configurable tile 305 to realize that the packet of databeing routed down its column configuration lines 325 is addressed to itstile 315, as shown in FIG. 5. Hence, as shown in this figure, the tileselector of tile 315 extracts the remaining data frames in the packet300.

In some embodiments described below, the column-identifying frame 305 isrouted down each column as it serves as (1) a column enable signal foreach column that contains an addressed tile, and (2) a column disablesignal for each column that contained a previously addressed tile. Insome of these embodiments, the tile-identifying frame 310 also is routeddown each column before the column select circuitry of the particularcolumn can determine that the particular column does not contain thedestination (i.e., addressed) tile for the current packet.

Some embodiments have different types of configurable tiles, such as oneor more configurable logic tile types, one or more configurable memorytile types, one or more configurable edge tile types (e.g., north,south, east, and west), etc. Some of these embodiments allow tiles to beindividually addressed, globally addressed (i.e., all addressedtogether), or addressed based on their tile types. Addressing multipletiles concurrently is beneficial because it allows concurrent reads fromor writes to multiple tiles. The tile types and the addressing of thesetile types will be further described in Section III below.

The configurable IC includes numerous user-design state elements (“UDSelements”) in some embodiments. UDS elements are elements that storevalues that at any particular time define the overall user-design stateof the configurable IC at that particular time. Examples of suchelements include latches, registers, memories, etc. The configurable ICof some embodiments might not include all such forms of UDS elements, ormight include other types of UDS elements.

In some embodiments, the configuration/debug network connects to some orall of the UDS elements (e.g., latches, registers, memories, etc.) ofthe configurable IC. In some embodiments, the configuration/debugnetwork has a streaming mode that can direct various circuits in one ormore configurable tiles to stream out their data during the operation ofthe configurable IC. Accordingly, in the embodiments where theconfiguration/debug network connects to some or all of the UDS elements,the configurable/debug network can be used in a streaming mode to streamout data from the UDS elements of the tiles, in order to identify anyerrors in the operation of the IC. In other words, the streaming of thedata from the UDS elements can be used to debug the operation of theconfigurable IC.

The streaming mode is used in some embodiments to form a logic analyzer,which may be on or off the same IC die that includes the configurabletiles. For instance, some embodiments include a trace buffer on the sameIC die as the configurable tiles. This trace buffer can then be used torecord the data that is output form one or more tiles during thestreaming mode operation of the configurable IC. In other words, thetrace buffer can be used to implement an “on-chip” logic analyzer inconjunction with the streaming mode operation of the IC. An “off-chip”logic analyzer can also be formed by using an off-chip trace buffer(i.e., a buffer that is not on the same die as the configurable IC)while using the streaming mode operation of the IC's configuration/debugnetwork.

Some embodiments also use the configuration/debug network to performcheckpointing operations. Checkpointing is a sub-operation of a debugoperation. The checkpointing process of some embodiments periodicallystops the configurable IC's operations (e.g., stops the IC's operationsevery few million cycles). At each stoppage of the IC's operations, thecheckpointing process uses the configuration/debug network to retrievethe configurable IC's state at that time (e.g., to retrieve the valuestored by each UDS element of the configurable IC at that time). Oncethis process has retrieved the configurable IC's state, it causes theconfigurable IC to resume its operations. When an error is detectedduring the debug operation (i.e., after a “crash”), a user or debuggingapplication then loads the mostly recently checkpointed IC state (i.e.,stored IC state) within the IC, and resumes the debug operation in amore deliberate manner (e.g., slower or under more supervision) in orderto identify the cause of the error.

In some embodiments, the configuration/debug network has a broadcastingmode that can direct various resources (e.g., memories, storageelements, etc.) in one or more configurable tiles to store the samedata. For instance, the broadcasting mode can be used to initialize thememory blocks in the configurable memory tiles.

Section II below provides an overview of the configurable tiles of someembodiments of the invention. Section III then provides a more detaileddiscussion of the packet data structure of some embodiments of theinvention. Next, Section IV provides a more detailed discussion of theconfiguration/debug network of some embodiments of the invention.

Section V then provides a more detailed discussion of the broadcastingmode operation of some embodiments of the invention. Section VI providesa more detailed discussion of the streaming mode operations of someembodiments. Section VII then describes the logic analyzer and tracebuffer functionalities of some embodiments. Section VIII describes thecheckpointing process of some embodiments of the invention. Section IXdescribes the high-speed nature of the configuration/debug network ofsome embodiments.

In the discussion above and below, many of the features of someembodiments are described by reference to a network that is used forboth configuration operations and debug operations. One of ordinaryskill in the art will realize that some embodiments might use thisnetwork only for debug operations or only for configuration operations.

II. Overview of Configurable Tiles

FIG. 6 illustrates the configurable circuit architecture of someembodiments of the invention. As shown in FIG. 6, this architecture isformed by numerous configurable tiles 605 that are arranged in an arraywith multiple rows and columns. In FIG. 6, each configurable tile is aconfigurable logic tile, which, in this example, includes oneconfigurable three-input logic circuit 610, three configurableinput-select interconnect circuits 615, and eight configurable routinginterconnect circuits 620. For each configurable circuit, theconfigurable IC 600 includes a set of storage elements for storing a setof configuration data.

In some embodiments, the logic circuits are look-up tables (LUTs) whilethe interconnect circuits are multiplexers. Also, in some embodiments,the LUT's and the multiplexers are sub-cycle reconfigurable circuits, asdescribed in U.S. patent application Ser. No. 11/082,193, filed on Mar.15, 2005. In some of these embodiments, the configurable IC is areconfigurable IC that stores multiple sets of configuration data forits sub-cycle reconfigurable circuits, so that reconfigurable circuitscan use a different set of configuration data in different sub-cycles.

In FIG. 6, an input-select multiplexer is an interconnect circuitassociated with the LUT 610 that is in the same tile as the input selectmultiplexer. One such input select multiplexer (1) receives severalinput signals for its associated LUT, and (2) based on itsconfiguration, passes one of these input signals to its associated LUT.

In FIG. 6, a routing multiplexer is an interconnect circuit that at amacro level connects other logic and/or interconnect circuits. Unlike aninput select multiplexer of some embodiments (which only provides itsoutput to a single logic circuit, i.e., which only has a fan out of 1),a routing multiplexer of some embodiments is a multiplexer that (1) canprovide its output to several logic and/or interconnect circuits (i.e.,has a fan out greater than 1), or (2) can provide its output to otherinterconnect circuits.

In some embodiments, some or all routing multiplexers can also serve aslatches. For instance, some embodiments use a complimentary passgatelogic (CPL) to implement a routing multiplexer. Some of theseembodiments then implement a routing multiplexer that can act as a latchby placing cross-coupled transistors at the output stage of the routingmultiplexer. Such an approach is further described in U.S. patentapplication Ser. No. 11/081,859, filed Mar. 15, 2005. In the discussionbelow, routing multiplexers that can serve as latches are referred to asrouting-circuit latches (“RCLs”).

In the architecture illustrated in FIG. 6, each configurable logic tileincludes one three-input LUT, three input-select multiplexers, and eightrouting multiplexers. Other embodiments, however, might have a differentnumber of LUT's in each tile, different number of inputs for each LUT,different number of input-select multiplexers, and/or different numberof routing multiplexers. Other embodiments might also use differenttypes of logic circuits and/or interconnect circuits. Several sucharchitectures are further described in the U.S. application Ser. No.11/082,193.

In some embodiments, the examples illustrated in FIG. 6 represent theactual physical architecture of a configurable IC. However, in otherembodiments, the examples presented in FIG. 6 topologically illustratethe architecture of a configurable IC (i.e., they show arrangement oftiles, without specifying a particular position of the circuits). Insome embodiments, the position and orientation of the circuits in theactual physical architecture of a configurable IC is different from theposition and orientation of the circuits in the topological architectureof the configurable IC. Accordingly, in these embodiments, the IC'sphysical architecture appears quite different from its topologicalarchitecture. For example, FIG. 7 provides one possible physicalarchitecture of the configurable IC 600 illustrated in FIG. 6. In FIG.7, sets of four tiles are aligned so that their LUT's are placed closerto each other. The aligned set of four tiles can be conceptually viewedas simply another tile itself.

FIGS. 6 and 7 illustrate only configurable non-edge logic tiles. Theconfigurable IC of some embodiments includes other types of configurabletiles, such as configurable edge logic tiles (“edge tiles”) andconfigurable memory tiles. In some of these embodiments, configurableedge tiles are similar to the configurable non-edge logic tiles of FIG.6, except that configurable edge tiles have additional configurablerouting circuits for routing input and output data to and from thecircuits in the configurable tile arrangement illustrated in FIGS. 6 and7.

On the other hand, configurable memory tiles (1) are tiles with blocksof memory, or (2) are tiles that are adjacent to blocks of memory. FIG.8 illustrates two examples of configurable memory tiles. The firstexample is a memory tile 805 that is formed by a set of four alignedtiles that have a memory block 810 in place of their four LUT's. In thesecond example, four aligned tiles 820 (which are formed by sixteentiles 815) neighbor a memory block 820. In the four aligned tiles 805and 820, the input select and routing interconnects serve asconfigurable ports of the memory blocks. In addition to the four alignedtiles 805 and 820, FIG. 8 illustrates several aligned sets of four logictiles 825, which are similar to the four aligned logic tiles of FIG. 7.

Even though FIG. 8 illustrates a particular tile architecture, one ofordinary skill will realize that other embodiments might use differenttile architectures. For instance, the architecture of some embodimentsincludes at least one memory array similar to the memory array 820,except that the array of these embodiments is surrounded by eightaligned tiles 820, four on each side of the array. Also, someembodiments have a different number of routing or input-selectmultiplexers. Some embodiments have different types of interconnectand/or logic circuits.

III. Packet Data Structure

Some embodiments use a configuration/debug controller to formulateconfiguration/debug packets, which are then routed to the configurabletiles of a configurable IC during configuration/debug operations. FIG. 9illustrates one such controller 910. This controller 915 formulatesconfiguration/debug packets and routes such packets to the configurabletiles 905 over a set of lines 910 that traverse each configurable tilesin a tile arrangement 900. The controller formulates configuration/debugpackets at a fast rate in some embodiments. In some embodiments, eachtile 905 in FIG. 9 corresponds to an aligned set of four tilesillustrated in FIG. 8.

In some embodiments, the set of lines 910 includes eighteen lines, sixof which are used to provide control signals, and twelve are used toprovide data signals. The six control signals serve as an opcode(operation code), while the twelve signals serve as the operand (i.e.,data argument) associated with the opcode. Accordingly, the six linesare referred to below as the opcode lines while the twelve lines arereferred to as the operand lines.

As mentioned above, some embodiments use a packet switching technologyto route data to and from the resources in the configurable tiles.Hence, over the eighteen lines that traverse through each set of tiles,these embodiments can route variable length data packets to configurabletiles sequential, randomly, or based on tile types (including a globaltile type).

FIG. 10 illustrates an example of a data packet 1000 that can be routedto one or more configurable tile sets. As shown in this figure, the datapacket 1000 includes several frames 1005. Each frame has a six-bitopcode 1010 and a twelve-bit operand 1015. In some embodiments,different data packets can have different number of frames. In otherwords, some embodiments allow the configuration/debug network to usevariable sized packets.

FIG. 11 illustrates examples of some opcodes that are used in someembodiments. These opcodes include:

-   -   (1) tile X, which has its lower eight bits identify the column        associated with a tile and its higher four bits identify a tile        type,    -   (2) tile Y, which has its lower eight bits identify the row        associated with a tile and its higher four bits identify a tile        type,    -   (3) Load Address, which provides an address for an addressed        tile to load in its loadable address counter,    -   (4) Read, which directs the addressed tile to provide the output        of a particular resource (e.g., storage elements, RCL, etc.)        that is identified by the address in the address counter,    -   (5) Read Increment, which directs the addressed tile to provide        the output of a particular resource that is identified by the        address in its address counter, and then to increment the        address in its address counter,    -   (6) Write, which directs the addressed tile to write to a        particular resource that is identified by the address in the        address counter,    -   (7) Write Increment, which directs the addressed tile to write        to a particular resource that is identified by the address in        the load address counter, and then to increment the address in        its address counter.

As shown in FIG. 11, the least significant eight bits of the operands ofthe tile X and tile Y opcodes provide the column and row address of atile. The four most significant bits of these two operands provide thetile types. These four bits provide (1) fourteen specific types, whichcan be used to collectively access tiles of the same type (e.g., accesssmall memory tiles or large memory tiles for initializing their storedcontent), (2) one global type, which specifies all of the tiles, and (3)one “No Type Select” type, which specifies that a set of tiles is beingindividually addressed (i.e., specifies that the set of tiles are beingaddressed by the least significant eight bits of the tile X or Yoperand, as opposed to being addressed based on their types). In someembodiments, the fourteen tile types include a logic tile type, a firstmemory tile type, a second memory tile type, an edge tile type, etc.

Because of the tile X and tile Y opcodes, the packets can access thetiles during configuration and/or debugging in any random access manner.More generally, the operands of the tile X and Y opcodes allow theconfiguration/debug network to access sets of tiles individually,globally, or based on their types. By allowing several tiles to beaddressed together, the tile type addressing allows simultaneous readsfrom or writes to resources in several tiles at once. For instance, atile X frame can specify a global tile type, and this frame can befollowed by a tile Y frame that specifies a global tile type or anothertile type. These two frames will result in the selection of severaltiles, which might be in different rows and columns. Such read and writeoperations are used during the streaming and broadcasting modes, whichare further described below in Sections V and VI.

As shown in FIG. 11, the least significant eight bits of the LoadAddress frame's operand can provide the address of a set of resourceswithin an addressed set of tiles. Examples of such resources include (1)storage elements that store configuration data, (2) RCLs (i.e., routingmultiplexers that can serve as latches), (3) storage elements (e.g.,latches and/or registers) that store mode bits that define one or moreoperational modes of the resources within the set of tiles, and (4)storage elements (e.g., memory cells) of a memory array.

For instance, when the eighth bit of this operand is a zero, the addressoperand in some embodiments provides the address of a set of storageelements (called configuration cells below) that store configurationdata for configurable circuits in the set of addressed tiles. In someembodiments, each set of configuration cells includes twelveconfiguration cells.

On the other hand, when the eighth bit is a one, the address operandprovides the address of (1) up to eight twelve-bit registers, or (2) oneof three sets of RCLs. Each twelve-bit register is a set of storageelements that store mode settings. In the example illustrated in FIG.11, the eight registers that store mode settings include:

-   -   (1) a mask storage that stores mask bits for masking the output        of the RCLs,    -   (2) a merge storage that stores merge bits for merging signals        on the twelve operand lines that traverse each column of tiles,    -   (3) for each set of edge tiles, four configuration control        storages for storing configuration data related to the        programmable inputs/outputs,    -   (4) two clock storages for storing clock configurations for the        set of tiles.

In some embodiments, each one of the three RCL addresses identifies adifferent set of twelve RCLs in the tiles. However, in some embodiments,different RCL addresses identify different number of RCLs. For instance,two of the addresses for the RCLs identify two sets of twelve RCLs,while a third address for the RCLs identifies a set of four RCLs, insome embodiments.

Some embodiments address small memory blocks by setting a bit in one ofthe clock control registers. This bit determines if the IC is in aconfiguration RAM mode (normal mode) or user RAM access mode. If the bitis set, reads and writes to up to 127 addresses associated with thelower seven bits in the address register are directed to locations in asmall memory block associated with an addressed tile. Since the clockcontrol registers are not in this range, the bit can be set and unset ineither mode.

For larger memory blocks (i.e., memory blocks with more than 128addresses), a bit is set in the adjacent tile. When this bit is set, thememory tile is in RAM access mode. If necessary, the width of theaddress register is increased to fully access the RAM. In someembodiments, the control bit is in a different tile, because the clockcontrol register in the memory tile cannot be accessed when in RAM mode.

The Read Increment and Write Increment frames are quite useful inreducing the number of frames routed through the configuration/debugnetwork. This is because these opcodes obviate the need to load a newaddress after each read or write. Accordingly, when reading or writingseveral sequential locations on the IC, the Read Increment or WriteIncrement opcodes cause a set of address counters in a set of addressedtiles to increment to the next address location for the subsequent reador write operation. One example of this will now be described byreference to FIG. 12.

FIG. 12 illustrates an example of a packet 1200 for a particular tile.As shown in this figure, the packet 1200 includes seven frames. Thefirst frame 1205 specifies the x-coordinate of an addressed tile, thesecond frame 1210 specifies the y-coordinate of the addressed tile, andthe third frame 1215 specifies the address of a set of resources withinthe addressed tile. The specified address is stored in an addressregister of the addressed tile in some embodiments. The next four frames1220-1235 in the packet 1200 then specify a read-increment operation,followed by a read operation, a write operation, and then a final readoperation.

The read-increment frame 1220 directs the addressed tile to provide theoutput value of the set of resources identified by the address (ADDR Ain this example) that was previously specified by frame 1215 and that isnow stored in its address register. As shown in FIG. 12, the operand ina read frame or read-increment frame is not relevant when theconfiguration/debug controller prepares such a frame. This is becausethe operand will be written over during the read operation. In otherwords, the output of the addressed set of resources is stored in theoperand section of the read frame, which like all frames of all packetsis routed out of configurable IC after being routed to a particular setof addressed tiles.

The read-increment frame 1220 also directs the addressed tile toincrement the address (i.e., ADDR A) in its address register by oneafter providing the output value of the set of resources addressed byframe 1215. The read frame 1225 then directs the addressed tile to readthe output value of the set of resources identified by the incrementedaddress (i.e., ADDR A+1) in the address register. The write frame 1230next directs the addressed tile to store the value (i.e., Value Q) inits operand section in the set of resources located at ADDR A+1. Theread frame 1235 then directs the addressed tile to read the output ofthe set of resources located at ADDR A+1.

The packet 1200 of FIG. 12 can be followed by another packet for a tilein the same column as the tile addressed by packet 1200. FIG. 13illustrates such a packet 1300. This other packet does not start with aframe that provides the x-coordinate of an addressed tile, as it isdirected to a tile in the same column as tile addressed by packet 1200.Instead, the packet 1300 starts with a frame 1305 that identifies they-coordinate of the newly addressed tile. This frame 1305 will notifythe previously addressed tile that its packet has ended and that thesubsequent frames are not intended for it. Also, the newly addressedtile will read the frame 1305 and know that the next set of frames 1310are intended for it, until it receives a new tile X or tile Y frame.

IV. Network Structure in Each Tile

FIG. 14 provides an overview of the configuration and debug network 1400of some embodiments. As shown in this figure, this networks includes aneighteen bit-wide bus 1405 and a configuration/debug controller 915. Theeighteen bit-wide bus 1405 passes through each tile of a configurable IC1410, so that the configuration/debug controller 915 can routeconfiguration/debug packets to the tiles of the configurable IC 1410.

The network 1400 also includes at least one eighteen bit-wide set ofstorage elements 1415, 1420, 1425, or 1430 in each tile. One such set ofstorage element exists at the boundary between each horizontally orvertically adjacent tiles. These sets of storage elements store (e.g.,latch) the data that they receive for a clock cycle. In someembodiments, each set of storage elements includes eighteen double-edgetriggered flip flops, each of which is formed by two multiplexed latchesthat latch on different edges of the clock. Such a flip-flop is furtherdescribed in U.S. patent application Ser. No. 11/292,952.

As shown in FIG. 14, the tile array includes four types of tiles, whichare: top, edge, central, and bottom. Central and edge tiles have asimilar circuit structure in the network 1400, except that edge tilesstore more configuration bits as they control the configurable I/Os ofthe configurable IC 1400 and may contain different programmableresources (e.g., the east/west tiles might contain LUTs, while thenorth/south tiles might not).

The top tiles have a network circuit structure that allows packets topass along the top tile row. The top tiles also include the columnselection functionality that can route a particular packet down aparticular column that is addressed. This column selection functionalityroutes tile X and tile Y frames down each column as well. The tile Xframe is routed down each column as it serves as (1) a column enablesignal for each column that contains an addressed tile, and (2) a columndisable signal for each column that contained a previously addressedtile. In the configuration/debug network 1400, the tile Y frame alsogets routed down each column before the column select circuitry of theparticular column can determine that the particular column does notcontain the destination (i.e., addressed) tile for the current packet.

The network circuit structure of each tile also includes a tileselection circuit that allows a tile to detect that a packet isaddressed to it. In addition, the bottom tiles have a network circuitstructure that allows the output of the different columns to be mergedinto one bus 1450 that traverses from left to right through the bottomtiles, and then loops back through the bottom tiles from right to leftto route the configuration/debug packets out of the tile array 1410, asshown in FIG. 14.

The bus 1450 loops horizontally through the bottom tiles, in order tooutput the configuration/debug packets to the configuration/debugcontroller 915 from the same side of the tile arrangement 1410 as thiscontroller supplies the packets to the tile arrangement. This simplifiesthe timing problem for determining when the configuration/debugcontroller 915 will receive the results of a configuration/debug packetthat it sends into the tile arrangement 1410. It allows the controller915 to send and receive packets at a frequency that is independent onthe size of the array. If the height of the array would cause asignificant delay for signals output from the bottom of the array toreach the controller at the top of the array, some embodiments mighthave the bus 1450 to loop back up through the tiles (e.g., through theleftmost tiles) so that it can output data in the same vicinity as thetile arrangement receives input.

In some embodiments, the bus 1450 is wider than eighteen bits wide(i.e., is wider than the eighteen bit column lines whose outputs aremerged into the bus at the end of each column). For instance, in someembodiments, this bus is thirty-six bits wide. Alternatively, in someembodiments, this bus is wider in one of its directions (e.g., left toright) than in its other direction (e.g., right to left).

The configuration/debug network 1400 has a fixed latency through each ofthe tiles. In other words, because of the synchronous set of storageelements at the boundary of each horizontally or vertically alignedtiles, two packets that are addressed to two different tiles have thesame delay from the input 1455 to the output 1460 of the tilearrangement 1410. This allows different packets to reach the same tileswithout interfering with one another. This guarantees that two differentread commands to the same tile do not interfere with each other.

In some embodiments, the configuration/debug network 1400 is completelyseparate network than the routing fabric network (i.e., the datanetwork) formed by the routing multiplexers and wiring connected tothese multiplexers that connects the configurable logic circuits whilethe IC operates. In other embodiments, the network 1400 shares somewiring and/or interconnect resources with the routing fabric, butincludes other resources that it does not share with the routing fabric.Example of resources that the network 1400 does not share in someembodiments includes the configuration/debug bus 1405, and the storageelements 1415, 1420, 1425, and 1430. Examples of routing fabric ofwiring and interconnects that connect the configurable logic circuitsare disclosed in U.S. patent application Ser. No. 11/082,193.

The network circuit structure in each of the tile types will now befurther described in sub-sections A-C.

A. Top Tiles

FIG. 15 illustrates the circuits of the configuration/debug network 1400in a top tile 1500. As shown in this figure, these circuits include two18-bit wide sets of storage elements 1505 and 1510, one tile X/tile Ydecoder 1515, an OR gate 1520, an AND gate 1525, a column selector 1530,a row selector 1535, two set and reset registers 1540 and 1545, aconceptual memory block 1560, masking logic 1550 and a multiplexer 1555.

As shown in this FIG. 15, the set of storage elements 1505 connects tothe 18-bit wide bus 1405 to receive 18-bit packet frames. On each edgeof the clock, the set of storage elements stores eighteen bits of data(i.e., a frame) that appears on the bus 1405 and outputs the eighteenbits of data (i.e., the frame) that it latched on the prior clock edge.Each set of eighteen bits (i.e., each frame) that the storage elementset 1505 outputs is routed to the next top tile along the eighteen bitwide bus 1405, as shown in FIG. 15. The eighteen bits output from thelast top tile are discarded in some embodiments.

The tile X/tile Y decoder 1515 receives the higher six bits of eacheighteen bits (i.e., each frame) that is output from the storage elementset 1505. This decoder examines these six bits to determine whetherthese six bits represent a tile X or tile Y opcode. If not, the decoder1515 outputs a “0”.

Alternatively, when the six bit opcode is a tile X or tile Y opcode, thedecoder 1510 outputs a “1”, which causes the OR gate 1520 to output “1”along its eighteen output lines. These outputs of the OR gate 1520, inturn, allow the eighteen bits that are output from the storage elementset 1505 to pass through the eighteen bit-wide AND gate 1525 (i.e.,cause the AND gate to output eighteen bits that are identical to theeighteen bits that it receives from the storage element set 1505).

The storage element set 1510 receives the eighteen bit wide output ofthe AND gate 1525. On the next clock edge, the storage element set 1510outputs the eighteen bits. The column selector 1530 receives the outputof the storage element set 1510. The column selector 1530 determineswhether the higher six bits represent a tile X opcode, and if so,whether the operand of the received tile X frame matches the type or thex-address of the tile 1500. As mentioned above, the lower eight bits ofa tile X frame provide the x-address (i.e., the column address) of atile, while its next four bits provide the type of the tile.

The column selector directs the register 1540 to assert a reset signal(i.e., a “0” in this case) when the column selector receives a tile Xframe (i.e., the sixteen bits output from the storage 1510) that has anoperand that matches neither the type nor the x-address of the tile1500. On the other hand, when the received frame is a tile X frame withan operand that matches the type or x-address of the tile 1500, thecolumn selector 1530 directs the register 1540 to assert a set signal(i.e., a “1” in this case). A set signal causes the OR gate's outputs toremain high even after the decoder 1510 pulls its output low when thisdecoder no longer detects a tile X or tile Y opcode (i.e., no longerreceives a tile X or tile Y frame). By keeping the OR gate outputs high,the AND gate 1525 continues to route frames down the column of tile1500, until the time that the column selector 1530 receives a tile Xframe whose operand does not match the type or the x-address of the tile1500. Once column selector receives such a tile X frame, it directs theregister 1540 to reset its output (i.e., to output a “0”). At thispoint, when the tile X/Y decoder does not output a “1”, the OR gate 1520will output a “0” (i.e., will prevent the AND gate 1525 from routing anymore frames down the column of tile 1500) until the tile X or Y decoder1515 detects another tile X or Y frame.

The row selector 1535 also receives the output of the storage 1530. Therow selector 1530 determines whether the received frame is a tile Yframe (i.e., whether the higher six bits output from the storage 1530),and if so, whether the operand of the received tile Y frame matches thetype or the y-address of the tile 1500. As mentioned above, the lowereight bits of a tile Y frame provide the y-address (i.e., the rowaddress) of a tile, while its next four bits provide the type of thetile.

The row selector directs the register 1545 to assert a reset signal(i.e., a “0” in this case) when it receives a tile Y frame with anoperand that matches neither the type nor the y-address of the tile1500. On the other hand, when the received frame is a tile Y frame withan operand that matches the type or the y-address of the tile 1500, therow selector 1535 directs the register 1545 to assert a set signal(i.e., a “1” in this case). A set signal from the register 1545activates the decoder logic associated with the conceptual memory block1560 of the tile 1500, while the reset signal from the register 1545deactivates this decoder logic.

The conceptual memory block 1560 conceptually represents tile 1500's (1)storage elements that store configuration data, (2) RCLs (i.e., routingmultiplexers that can serve as latches), (3) storage elements (e.g.,latches and/or registers) that store mode bits that define one or moreoperational modes of the resources within the tile 1500, and (4) storageelements (e.g., memory cells) in a memory array.

The storage elements and RCLs are not actually organized in a contiguousblock in some embodiments. However, in some embodiments, groups ofstorage elements (e.g., configuration cells), RCLs, and/or registers canbe enabled at one time for a read or write operation. For instance, insome embodiments, groups of twelve configuration storage elements,groups of twelve or four RCLs, and groups of twelve one-bit registerscan be enabled at one time. The decoder logic for addressing thesegroups of storage elements and RCLs will be further described below.

The conceptual memory block also receives the eighteen-bit output of thestorage 1510 (i.e., connects to the eighteen bit-wide bus 1405 andreceives the frame output from the storage 1510). Through thisconnection, data can be written to the RCLs and storage elements (e.g.,configuration, register, and memory cells) in the block 1560.

U.S. patent application Ser. No. 11/081,859 discloses the RCL design ofsome embodiments of the invention. As disclosed in this application,each RCL is an n-to-one multiplexer (where n is any integer greaterthan 1) that has a complementary pass logic design. As shown in FIG. 16,this multiplexer also has two output buffers 1605 and 1610 that arecross coupled by two transistors 1615 and 1620 (i.e., one transistorconnects the input of first buffer to the output of the second buffer,while the other transistor connects the input of the second buffer tothe output of the first buffer). These two transistors when enabledcause the output stage of the multiplexer to form a latch. To write tosuch an RCL, some embodiments insert a write-enable circuit 1625 inseries with one of the cross-coupling transistors, as shown in FIG. 16.This figure illustrates that in some embodiments the write-enablecircuit includes one NMOS transistor 1635 and one PMOS transistor 1630.The NMOS transistor 1635 is in series with one 1615 cross couplingtransistor. The PMOS transistor 1630 connects at its drain to the nodebetween the two transistors 1615 and 1635, and connects at its source tothe value that needs to be written into the latch. The gates oftransistors 1630 and 1635 are both tied to the complement of the Writesignal, which is high when a value has to be written into the latch.When the Write signal is high, the transistor 1635 is off, and thetransistor 1630 is on to pull the node between the two transistors 1615and 1635 to the value being written, which in turn places the output ofthe buffers to the desired values.

Data can be read from (1) the storage elements (e.g., configuration,register, and memory cells), and (2) the RCLs in the block 1560 throughn address lines 1565, where n is an integer larger than eleven in someembodiments. These lines are fed to a multiplexer 1555 through themasking logic 1550.

The masking logic 1550 also receives the lower twelve-bit output of thestorage 1510 (i.e., the bit lines for the frame's operand). The maskinglogic can replace some of the bits output from the memory block 1560 onlines 1565 with the operand bits being output from storage element 1510.This masking logic, in conjunction with merging logic in the bottomtiles, allows bits to be read from different tiles in potentiallydifferent rows and columns at the same time. This masking logic isfurther described below.

The multiplexer 1555 selects between the potentially masked twelve-bitoutput from the masking logic 1550 (i.e., the memory-block data path)and the lower twelve-bit output of the storage 1510 (i.e., thepacket-frame data path). The multiplexer selects the memory-block datapath during a read operation, while it selects the frame-operand datapath during a write operation. As mentioned above and further describedbelow, the data from the frame-operand data path can be introducedthrough the masking logic into the memory-block data path and outputduring a read operation. As shown in FIG. 15, the twelve-bit output ofthe multiplexer 1555 merges with the higher six-bit output of thestorage 1510 onto the eighteen-bit bus 1405, which is routed to a tilebelow the top tile 1500.

FIG. 17 illustrate the decoder logic 1700 of memory block 1560. As shownin this figure, the decoder logic includes two decoders 1705 and 1715and an address counter. The decoder 1705 receives the output of theset/reset register 1545 and the six bit opcode output (i.e., the opcodeof the frame output) from the storage 1510. When the output of theregister 1545 is active (i.e., is set), the decoder 1705 decodes theopcode that it receives to determine whether to assert a read signal, awrite signal, a load address, and/or an increment address on its output.

The decoder 1705 asserts a read signal when the opcode specifies a reador read increment. It asserts a write signal when the opcode specifies awrite or write increment. It asserts a load address when it received aLoad Address opcode. It asserts an increment address signal after itreceives a Read Increment or Write Increment opcode and it causes a reador write operation to be performed. The load address and incrementaddress signals are supplied to the address counter 1710. The addresscounter 1710 also receives the twelve-bit operand of the frame output(i.e., within the eighteen bit output) of the storage 1510. When theload address signal is active (i.e., is asserted by the decoder), theaddress counter loads in the address specified by the twelve-bitoperand. Alternatively, when the increment address signal is active, theaddress counter increments the address that is currently stored in theaddress counter.

The address counter outputs the address that it stores to the seconddecoder 1715, which is responsible for enabling a set of blocks 1720that represent storage elements (e.g., the configuration cells, registercells, memory cells, etc.) and RCLs of the tile 1500. Each address thatthe decoder block receives can identify up to twelve storage elements(e.g., configuration cells, register cells, memory cells, etc.) or RCLs.As shown in FIG. 17, the decoder 1715 connects the blocks 1720 toseveral enable lines 1725 that allow the decoder 1715 to enable blocksthat are addressed by the address outputted from the address counter1710. As shown in this figure, one set of blocks 1720 can share oneenable line.

FIG. 17 also shows certain blocks 1720 receiving a write signal from thesecond decoder 1715, which corresponds to the write signal generatedfrom the first decoder 1705. These blocks represent the configurationcells, register cells, memory cells, RCLs, etc. that can store data.When the write signal is active, the twelve bit operand data is writtento the blocks enabled by the second decoder 1715. During a readoperation, the data from the enabled blocks (i.e., blocks enabled by thesecond decoder) is written for output onto the output lines 1565 of thememory 1560.

FIG. 18 conceptually illustrates the masking logic 1550 of the top tile1500 for some embodiments of the invention. As shown in this figure, themasking logic 1550 includes a mask register 1805, a multiplexer 1810,two AND gates 1815 and 1820, and an OR gate 1825. The multiplexer 1810,the AND gates 1815 and 1820, and the OR gate 1825 are twelve bits wideeach.

The multiplexer 1810 receives several sets of twelve output lines. Eachset of twelve output lines provides up to twelve outputs from RCLs andstorage elements (e.g., configuration cells, register cells, memorycells, etc.) in the conceptual memory block 1560 of the tile 1500. Insome embodiments, the masking is performed only for RCL outputs. Hence,in these embodiments, the input sets into the multiplexer 1810 only comefrom RCLs. However, in these embodiments, another multiplexer is used tocircumvent the masking logic when the RCL outputs are not beingsupplied.

The multiplexer 1810 routes the signal on one of its input sets to theAND gate 1820 based on an address signal that it receives on its selectline. This address signal can be generated by the decoder logic 1700.The AND gate 1820 also receives the output of the twelve bit maskregister 1805.

The mask register 1805 contains the masking data, which can mask(eliminate) certain data bits output from the memory block 1560 whileletting other data bits through. The masking data is written into themask register before the operation of the masking logic. The output ofthe masking register is also inverted and then supplied to the AND gate1815. The AND gate 1815 also receives the operand of the received frame(e.g., for tile 1500, the lower twelve bits output from the storage1510). The twelve-bit wide outputs of the two AND gates are supplied tothe OR gate 1825, which performs an OR function on these two outputs androutes their results to the multiplexer 1555.

When the output of the memory block is not to be masked, the maskingregister contains all “1's”, which results in the AND gate 1820 passingthrough all the signals output by the multiplexer 1810 and the AND gate1815 not passing through any of the signals on the bus 1405. On theother hand, when the output of the memory block is to be masked, themasking register contains a particular pattern of “1's” and “0's” thatresults in the AND gate 1815 and 1820 passing through a desiredcombination of bits from the bus 1405 and the memory block 1560.

Essentially, the two twelve-bit wide AND gates 1815 and 1820 and thetwelve-bit wide OR gate 1825 form a twelve-bit wide two-to-onemultiplexer. This multiplexer receives for its two twelve-bit inputs thetwelve-bit output of the multiplexer 1810 and the twelve-bit output ofthe storage 1510. The twelve-bit select lines of this multiplexerreceives the output of the twelve-bit mask register. Each mask bit valuethen selects between the corresponding bit value from the output of themultiplexer 1810 and the corresponding bit value from the output of thestorage 1510.

As mentioned above, the output of the OR gate 1825 is supplied to themultiplexer 1555. If the masking logic is not performed for the memoryblock 1560 in its entirety, a multiplexer is used to determine whetherto route the output of the OR gate 1825 or the output of the memoryblock 1560 to the multiplexer 1555.

Also, as mentioned above, the configuration/debug network of someembodiments is used in a reconfigurable IC (e.g., a sub-cyclereconfigurable IC). In some such embodiments, the mask register 1805stores different mask values (e.g., different twelve bit mask values)for different reconfiguration cycles (e.g., different sub-cycles) of thereconfigurable IC. In this manner, different masking operations can beperformed in different reconfiguration cycles (e.g., differentsub-cycles) to maximize the number of bits that are read from differenttiles.

Even though the top tile structure was described above by reference toseveral conceptual examples illustrated in FIGS. 15-18, one of ordinaryskill will realize that other embodiments might use different circuitsin the top tile. For instance, instead of using the AND gates 1815 and1820 and the OR gate 1825, some embodiments use an alternative circuitstructure to form a two-to-one multiplexer.

Also, FIG. 18 illustrates a multiplexer 1810 to describe conceptuallythe concept of a multiplexer that selects between various outputs of thememory block 1560. One of ordinary skill will realize that otherembodiments might not utilize an actual multiplexer structure, butinstead use a tri-state approach. For instance, several different setsof storage elements or RCLs might share a particular set of twelve-bitlines to provide their output. To do this, each set of storage elementsor RCLs has a set of tri-stateable driver that outputs their storedvalue onto the particular set of twelve bit lines. When a set of storageelements or RCLs are not being read, its associated set of drivers aretri-stated. On the other hand, when the set is being read, itsassociated set of drivers are used to drive the stored values of the setonto the particular set of twelve-bit lines.

B. Center and Edge Tiles

FIG. 19 illustrates the network structure of a center tile 1900. Asshown in this figure, the network structure of the center tile isidentical to the top tile, except that it does not include the storage1505, the tile X/tile Y decoder 1510, an OR gate 1520, the AND gate1525, the column selector 1530, the set/reset register 1540. The centertile basically includes all the circuitry necessary for determiningwhether a packet is intended for it, and if so, to perform theappropriate read, write, and mask operations.

The network structure for an edge tile is similar to the networkstructure for a center tile. The one difference between edge and centertiles is that, in some embodiments, the edge tiles have more storageelements (e.g., configuration or register cells) to deal with theconfigurable I/O functionalities of the tile arrangement 1400.

C. Bottom Tiles

FIG. 20 illustrates the network structure of a bottom tile 2000. Asshown in this figure, the network structure of the bottom tile issimilar to the top tile. Like a top tile, a bottom tile includes an18-bit wide storage 1510, a column selector 1530, a row selector 1535,two set and reset registers 1540 and 1545, a conceptual memory block1560, masking logic 1550 and a multiplexer 1555. However, it alsoincludes multiplexer 2005, storage elements 2010 and 2015, and a mergingcircuit 2025. Also, in the embodiment shown in FIG. 20, the columnselector 1530 and its associated register 1540 is after the multiplexer1555 not before. However, in other embodiments, the column selector 1530and its associated register 1540 can be placed before multiplexer 1555.

The operation of the row selector 1535 and its register 1545 isidentical in the bottom, center, and top tiles. The operation of thecolumn selector 1530 and its register 1540 are also similar in the topand bottom tiles. However, in the bottom tile, the column selector 1530and its register 1540 are used to merge the outputs of multiple columnsonto the bus 1450 of the configuration/debug network 1400, as furtherdescribed below.

The bus 1450 includes two sets of lines 2050 and 2055, the merging logic2025, and two sets of storage elements (e.g., two registers) 2010 and2015. The two sets of lines 2050 and 2055 form a horizontal output loopthrough the bottom tiles. The first set 2050 traverses from left toright, while the second set 2055 traverses from right to left. In abottom tile 2000 that is not on the left or right edge of the tilearrangement, the first set 2050 routes signals in from a bottom tile tothe left of the tile 2000 and route signals out to a bottom tile to theright of the tile 2000, while the second set 2055 route signals in fromthe bottom tile to the right of tile 2000 and route signals out to thebottom tile to the left of the tile 2000.

In some embodiments, the first set of lines 2050 include thirty-six datalines to allow each of the twelve data lines output from the multiplexer1555 to be carried on up to three data lines in the set 2050, as furtherdescribed below. The first set of lines 2050 connects to one set ofinputs of the multiplexer 2005, whose set of outputs is supplied to thestorage 2010. Thirty-six data lines 2065 from the first set 2050 is alsofed to the merging circuit 2025.

The merging circuit 2025 performs an analogous operation to the maskingcircuit 1550, but it performs this operation to facilitate the mergingof the data from the various columns, instead of the tiles from thevarious rows. Even when the bottom tile is selected, the merging logiccan replace some of the bits output from the multiplexer 1555 with thedata bits coming from the tile to the left of the bottom tile.

Specifically, the merging circuit 2025 has thirty-six output data lines2060, each corresponding to a particular data line in the thirty-sixdata lines 2065 that the circuit 2025 receives. The merging circuit 2025can place the signal on each output data line of the multiplexer 1555 onup to three of its output data lines 2060. On each particular outputdata line 2060, the merging circuit 2025 can also place a signal fromits corresponding input data line 2065.

The merging circuit 2025 determines the signal to place on an outputdata line based on a merge bit value, which causes the merge circuit toselect between an input data line 2065 and a data line from the outputof the multiplexer 1555. In some embodiments that are implemented in areconfigurable IC, the merge logic includes a merge bit for each outputline 2065 for each reconfiguration cycle (e.g., each sub-cycle). Also,to perform the selection between input data lines 2065 and data linesoutput from the multiplexer 1555, the merging circuit 2025 can utilize atwo-to-one multiplexer structure, such as the one formed by the ANDgates 1815 and 1820 and the OR gate 1825 of the masking circuit 1550.

The multiplexer 2005 has two sets of input lines and one set of outputlines 2070. As one of its sets of input lines, the multiplexer 2005receives the output lines 2060 of the merging circuit 2025. As its otherset of input lines, the multiplexer 2005 receives the first set of lines2050. As shown in FIG. 14, the multiplexer 2005 of the leftmost bottomtile receives a grounded first set of lines 2050.

The multiplexer 2005 has a select line that receives the output of theset/reset register 1540. When the register 1540 outputs a reset signal(in this case a “0”), the multiplexer selects the first set of lines2050 for output on its set of output lines 2070 (i.e., routes thesignals on the first set of lines 2050 to its output lines 2070).Alternatively, when the register 1540 outputs a set signal (in this casea “1”), the multiplexer selects the output lines 2060 of the mergingcircuit 2025 for output on its set of output lines 2070 (i.e., routesthe signals on the output lines 2060 to its output lines 2070).

The register 1540 assert a reset signal (i.e., a “0” in this case) whenthe column selector directs it to do so after the selector receives atile X frame (i.e., receives the eighteen bits output) from the storage1510 that has an operand that matches neither the type nor the x-addressof the tile 2000. On the other hand, when the received frame is a tile Xframe with an operand that matches the type or x-address of the tile2000, the column selector 1530 of tile 2000 directs the register 1540 toassert a set signal (i.e., a “1” in this case).

Accordingly, the multiplexer 2005 selects between the data path comingfrom the left of the bottom tile 2000 and the potentially masked/mergedoutput of the multiplexer 1555. The multiplexer 2005 selects thepacket-frame data path coming from the left side when the bottom tile'scolumn is not selected, while it selects the potentially masked/mergedoutput of the multiplexer 1555 when the bottom tile's column isselected. Through the merging operation of the merging circuit 2025, thedata from the data path coming from the left side can be introduced intothe data path from the tile's column even when the column is selected.This merging logic, in conjunction with masking logic in the each tiles,allows bits to be read from different tiles in potentially differentrows and columns at the same time.

The bottom tile contains two sets of storage elements 2010 and 2015 inits output path. The storage 2010 is in a forward direction out of thebottom tile 2000. This storage is there for retiming the signals at theboundary of two tiles. The other storage 2015 is in the loop backdirection through the bottom tile. It also performs retiming at theboundary of two tiles. However, the storage 2015 is primarily used forthe loop back through the tile arrangement, which as mentioned aboveallows configuration/debug packets to be output from the arrangement onthe same side that they are input. This allows the controller 915 tosend and receive packets at a frequency that is independent on the sizeof the array. To establish the loop back, the first set of lines 2050,or a portion of these lines, connects to the second set of lines 2055 inthe rightmost bottom tile.

V. Broadcasting

The configuration/debug network described above has the ability to writedata (e.g., configuration data sets or initialization data)simultaneously to various storage elements (e.g., various configurationcells or memory cells) in various different tiles. This ability ishighly advantageous for decreasing the time for configuringconfiguration cells (i.e., storing configuration data) or initializingmemory cells in a configurable IC.

FIG. 21 illustrates a process 2100 that the configuration controller 915can perform to operate the configuration/debug network in a broadcastingmode. As shown in this figure, the broadcasting process initially sends(at 2105) a tile X opcode that specifies the address (i.e., thex-coordinate or tile type) of all tiles or a subset of tiles. At 2110,the process sends a tile Y opcode that specifies the address (i.e., they-coordinate or tile type) of all tiles or a subset of tiles. The X andY opcodes together specify all the tiles or a subset of two or moretiles.

At 2115, the process sends a load address frame that provides an addressthat the addressed tiles need to store in their address counters. Thisaddress is the address of resources within each addressed tile to whichdata needs to be written. Next, at 2120, the process sends a Write orWrite Increment frame that contains twelve bits that the addressed tilesneed to store in their resources that are located at the address storedin their address counter. When the Write Increment frame is sent, theaddressed tiles increment the address in their address counter afterwriting to the specified-address

After 2120, the process determines (at 2125) whether it has broadcastedall the required Write or Write Increment frames. If so, the processends. Otherwise, the process determines (at 2130) whether it needs tochange the tile address for the next write operation that it needs tobroadcast. If so, the process determines (at 2135) whether it needs tochange the x-address of the tiles to which it needs to broadcast. If itdoes, the process returns to 2105 to send out another tile X frame. Ifit does not need to change the x-address of the addressed tiles, theprocess returns to 2110 to send out another tile Y frame. From 2105 and2110, the operation of the process 2100 is as described above.

When the process determines (at 2130) that it does not need to changethe tile address, the process transitions to 2140, where it determineswhether it needs to load a new address. Typically, the process would notload a new address for the same set of addressed tiles if its previouswrite operation was a Write Increment operation. If the processdetermines (at 2140) that it should load a new address, it transitionsback to 2115 to send out a new Load Address frame. Otherwise, theprocess transitions back to 2120 to send another Write or WriteIncrement frame. From 2115 and 2120, the operation of the process 2100is as described above.

VI. Streaming

In some embodiments, all UDS elements of the configurable IC areavailable on the configuration/debug network 1400. Examples of such UDSelements are RCLs, memory cells, and register cells, which connect tothe configuration/debug network through the circuit structuresillustrated in FIGS. 15-20. As the UDS elements are accessible throughthe configuration/debug network, this network can be used to access(read from or write to) the UDS elements in any sequential or randomaccess manner. Random access in this context means that the UDS elementscan be accessed through the configuration/debug network and the datapackets in any order desired by a user or debugger.

Moreover, as the UDS elements are accessible through theconfiguration/debug network, this network can read out the state (e.g.,the value of the RCLs, the memory cells, register cells, etc.) of theconfigurable IC while the IC is operating. This ability is highlyadvantageous for performing debugging during the operation of theconfigurable IC.

In some embodiments, the configuration/debug network has a streamingmode that can direct various resources (e.g., UDS elements) in one ormore configurable tiles to stream out their data during the user-designoperation of the configurable IC at the user design operating frequencyor faster. This streaming data makes the debugging abilities of theconfigurable IC even more robust as it allows a large amount of computedand configuration data to be output repetitively at a very fast rate.

FIG. 22 illustrates a process 2200 that the configuration controller 915can perform to operate the configuration/debug network in a streamingmode. As shown in this figure, the streaming process 2200 initiallyaddresses (at 2205) a set of tiles. The process can address such a setby sending a tile X frame followed by a tile Y frame that identify onetile. Alternatively, the process can address a set of two or more tilesby sending a tile X frame that specifies a global type (in order toenable the column selection circuit of each column) followed by a tile Yframe that specifies the tile type or tile address that identify thetile or tiles being addressed.

Next, the process 2200 sets (at 2210) the mask and merge bits in themask and merge registers of mask and merge logics 1550 and 2025 of theset of tiles addressed at 2205. The mask and/or merge bits mask out thevalues that are read from the UDS elements of the addressed set of tileswhen this set does not include any UDS element whose value has to bestreamed out during the streaming operation. Alternatively, when theaddressed set of tiles includes a particular set of user-design statesthat needs to be streamed out, the mask and/or merge bits do not maskout the values that are read from the UDS elements that need to bestreamed out.

At 2215, the process then provides a Load Address frame that identifiesthe resources in the set of addressed tiles (i.e., the set of tilesaddressed at 2205) that need be read during the streaming mode operationat 2230 and 2235. The Load Address provides the address (in the set ofaddressed tiles) of the resources (e.g., UDS elements) that need to beread during the streaming mode. When the set of addressed tiles does notinclude any UDS elements that are being read during the streaming mode,the Load Address frame can provide the address of any set of resourcesthat need to be masked in the set of addressed tiles.

After 2215, the process determines (at 2220) whether it needs to set themask and merger register values in any other set of tiles. If so, theprocess returns to 2205, which was described above. Otherwise, theprocess notifies (at 2225) all tiles that the subsequent set of readoperations are directed to them. In some embodiments, the process sonotifies the tiles by sending a tile X frame that specifies a globaltype (in order to enable the column selection circuit of each column)followed by a tile Y frame that specifies the global tile type. At 2225,the process also starts the user-design operation of the IC. In someembodiments, the user-design operation of the IC, might have beencommenced before the process 2200 of FIG. 22 even started. In otherwords, some embodiments allow tiles to be configured or reconfigured forstreaming after the user-design operation of the IC has started. Variousdefinitions for starting the user-design operation of a configurable ICare provided in Section X below.

At 2230, a Read frame is sent, which causes all tiles to read theaddress locations specified by the addresses that were set at 2215. Thisread out data is initially stored in the operand field of the Readframe. While outputting this data, the tiles mask and merge logiccircuits 1550 and 2025 eliminate the data bits that are supposed to bemasked out from the data stream that is streamed out of the tilearrangement. As further described below, the data stream can stream intoa trace buffer that is outside of the tile arrangement but on the sameIC die, or it can stream into a trace buffer that is outside of the tilearrangement and the IC die. As mentioned above, in some reconfigurableembodiments, the merge register can be set per each bit for eachreconfiguration cycle to allow the merging operation to be defined perbit per each reconfiguration cycle, while the mask register can be setper each reconfiguration cycle to allow the masking operation to bedefined per each reconfiguration cycle.

After 2230, the process determines (at 2235) whether it needs tocontinue the streaming mode by sending another Read frame in the nextcycle. If so, another Read frame is sent at 2230. In some embodiments,the process 2200 sends Read frames through the configuration network atsuch a rate to ensure that UDS data streams out of the IC at theuser-design operational rate or faster, e.g., at the sub-cyclereconfiguration rate in case of a sub-cycle reconfigurable IC. Forinstance, in some embodiments, the configurable IC is a reconfigurableIC that operates at 800 MHz to implement a 200 MHz user design withreconfigurable circuits that each loop through four configuration datasets in each user design cycle (i.e., that can reconfigure up to fourtimes in four sub-cycles associated with each user design cycle). Insuch an embodiment, the process 2200 could send Read frames through theconfiguration/debug network at a rate of 800 MHz to stream out UDSvalues at a rate of 800 MHz. In this manner, the monitored UDS valuescan be streamed out for the four sub-cycles in each user design cycle,which thereby provide the monitored UDS values for each user designcycle. The Read frames are repeatedly sent out until a determination ismade (at 2235) that the streaming mode should terminate. At this stage,the streaming process ends.

VII. Trace Buffer and Logic Analyzer

The streaming operation of the configuration/debug network 1400 can beused to create a logic analyzer functionality on the configurable IC. Insome embodiments, a logic analyzer has three components: (1) samplingcomponent, (2) a capture component, and (3) a trigger component. Thestreaming operation can serve as the sampling component of logicanalyzer. It can continuously provide samples of certain states of theconfigurable IC during the IC's operation.

An on-chip trace buffer can perform the capture component of the logicanalyzer. FIG. 23 illustrates an example of such a trace buffer 2310.The trace buffer 2310 is on the same configurable IC die 2300 as thetile arrangement 1400 and configuration controller 915. This bufferreceives the first set of lines 2050 of the bottom bus 1450 of theconfiguration/debug network 1400. As mentioned above, the lines 2050 insome embodiments are thirty-six bits wide, which allow the trace bufferto store thirty-six bits of streamed out data from the tile arrangement1410 on each clock cycle (e.g., each user design cycle). When the tilearrangement is part of a sub-cycle reconfigurable IC, the trace buffercan receive and store thirty-six bits on each sub-cycle of the userdesign cycle. The trace buffer is a circular buffer that continuouslystores the data that it receives until instructed otherwise. When thetrace buffer become full while recording, it will continue recording bystoring newly received data over the oldest data that it has recorded atthe time it receives the new data.

The trigger component of the logic analyzer is performed by a triggercircuit 2315 that communicates with the trace buffer 2310. This triggercircuit 2315 analyzes the data as it is being stored in the tracebuffer. When the trigger circuit 2315 identifies a particular set ofvalues or sequence of values that have been stored in the trace buffer,the trigger circuit directs the trace buffer to stop storing the datathat is being streamed out of the tile arrangement 1410. In this manner,the trace buffer will store a relevant subset of data that it receivedfor a certain time interval before it stored the trigger-event data thatthe trigger circuit detected. After stopping the trace buffer'srecording, the trigger circuit in some embodiments directs theconfiguration controller to stop the streaming mode operation of thetile arrangement (e.g., to stop sending Read frames).

FIG. 24 illustrates a logic analyzer process performed by the IC 2300 ofsome embodiments. As shown in this figure, this process initially starts(at 2405) the streaming mode operation of the IC. To do this, theprocess 2400 runs the process 2200 of FIG. 22. Next, the processcontinuously stores (at 2410) in the trace buffer 2310 the data beingstreamed out of the tile arrangement on lines 2050. Finally, the process2400 terminates (at 2415) the recording when the trigger circuit detectsthe trigger event while it is continuously analyzing the data stored inthe trace buffer. At 2415, the process also directs the configurationcontroller to terminate the streaming mode operation. At 2415, theprocess might also set a flag or generates a signal that signifies thedetection of the trigger event. This flag or signal is used in someembodiments to interrupt the operation of the configurable IC so that anexternal circuit can note and analyze the trigger event. Alternatively,this flag or signal does not result in the interruption of the operationof the configurable IC, but serves to simply notify a user or anothercircuit of the need to analyze or output the data in the trace buffer ata later time.

The logic analyzer functionality of the configurable IC 2300 of FIG. 23is unique. Existing configurable ICs cannot implement a logic analyzerfunctionality without altering the user's design. For instance, otherconfigurable ICs take the RTL (register transfer language) descriptionof the user's design and the RTL of a logic analyzer, and define aconfiguration bit stream that configures the configurable IC toimplement the combined design and logic analyzer RTLs. This requires thedesign to be changed. Also, if the right internal nodes were not pickedfor tracking, a new configuration bit stream needs to be generated tofactor a new RTL for the logic analyzer (which now examines differentnodes) and the configurable IC has to be configured again (if possible)with the new configuration bit stream. The configurable IC 2300 does notneed configuration data to be generated to implement the logic analyzerfunctionality, as its logic analyzer functionality is built into theconfiguration/debug network. In addition, the nodes that are analyzedfor the logic analyzer functionality in the configurable IC 2300 can beeasily modified by repeating the process 2200 and modifying (at 2215 ofthe process 2200 of FIG. 22) the resources that stream out theiroutputs.

FIG. 24 conceptually illustrates the logic analyzer functionality ofsome embodiments of the invention. Other embodiments might implementthis functionality differently. For instance, FIG. 24 illustrates aprocess where the detection of the trigger event terminates therecording of the UDS data. In other embodiments, however, the detectionof the trigger event starts the recording of the UDS data for aparticular interval of time or until the occurrence of another triggerevent. Also, in some embodiments, the occurrence of the trigger eventdoes not automatically start or stop the recording of the UDS values.Instead, after the trigger even occurs in these embodiments, therecording of the UDS values starts or stops only if certain operationalcondition exists (i.e., only under certain operational conditions). Forinstance, in some embodiments, the trigger event will cause therecording to start or stop if a write signal is present on a particularbus that is being monitored.

The streaming mode operation can also be used in conjunction with tracebuffers that reside outside of the configurable IC. FIG. 25 illustratesone such trace buffer 2510. Specifically, this figure illustrates atrace buffer 2510 that is not on the same IC die 2500 as the tilearrangement 1410 and configuration/debug controller 915. The advantageof placing a trace buffer off chip is that there are less restrictionson the size of the trace buffer when it is not on the same die as theconfigurable IC. On the other hand, the disadvantage of placing a tracebuffer off chip is that such a location consumes some of theinput/output resources of the IC. This consumption is often amelioratedby routing fewer signals to the external trace buffer. For instance,while the trace buffer 2310 of FIG. 23 might receive thirty-six bits pereach cycle, the trace buffer 2510 of FIG. 25 might receive a fraction ofthirty-six bits per each cycle.

The on-chip or off-chip logic analyzer functionality is useful fordebugging (i.e., detecting faulty operations) of the configurable IC. Itis also useful for collecting statistical data regarding the operationof the configurable IC. It is further useful for verifying correctoperation of the IC.

VIII. Check Pointing

Another advantage of having all user states on the configuration/debugnetwork is that this availability allows the network to performcheckpointing operations without altering the state of the IC. FIG. 26illustrates a debugging process 2600 that uses the configuration/debugnetwork of some embodiments to perform checkpointing.

As shown in this figure, the process initially starts (at 2605) theuser-design operation of the IC. Next, the process stops (at 2610) theuser-design operation of the IC after the IC has operated a particularnumber of cycles. Section X provides several different examples ofstarting and stopping user-design operations of the IC.

Through the configuration/debug network, the process then retrieves (at2615) the state of each UDS element (i.e., RCL, memory cell, registercell, etc.) of the configurable IC. The process accesses each UDSelement by stepping through the tiles one at a time (e.g., by using tileX and/or Y frames) and successively reading sets of UDS elements in eachtile (e.g., by using Read or Read Increment frames). The process 2600does not need to step through the tiles in any particular manner, andinstead can access the UDS elements in a random access manner. In someembodiments, the process stores (at 2615) each retrieved state of theconfigurable IC in a memory outside of the configurable IC.

After 2615, the process re-starts (at 2620) the user-design operation ofthe IC. In re-starting the operation of the IC, the inputs that theprocess provides to the IC are the inputs that the IC is suppose toreceive in the clock cycle that is after the clock cycle that was lastcheckpointed (i.e., after the clock cycle whose state was checkpointedat 2615).

At 2625, the process determines whether any error has been encounteredbefore reaching the next checkpointing milestone. If not, the processstops (at 2630) the user-design operation of the IC after the IC hasoperated another particular number of cycles. Through theconfiguration/debug network, the process then retrieves (at 2635) thestate of each UDS element (i.e., RCL, memory cell, register cell, etc.)of the configurable IC. The process accesses each UDS element bystepping through the tiles one at a time (e.g., by using tile X and/or Yframes) and successively reading sets of UDS elements in each tile(e.g., by using Read or Read Increment frames). As at 2615, the processin some embodiments stores (at 2635) each retrieved state of theconfigurable IC in a memory outside of the configurable IC.

After 2635, the process determines (at 2640) whether it has finished itsdebugging operation. If so, the process ends. If not, the processreturns to 2620.

When the process encounters an error (at 2625) before reaching the nextcheckpointing milestone, the process stops the operation of the IC at2645. At 2645, the process then uses the configuration/debug network toload the last checkpointed state of the configurable IC. In other words,the process uses the configuration/debug network to write to the IC'sUDS elements, such as its RCLs, memory cells, register cells, etc.Through the configuration/debug network, the process (at 2645) stepsthrough the tiles one at a time (e.g., by using tile X and/or Y frames)and successively writes to sets of UDS elements in each tile (e.g., byusing Write or Write Increment frames).

After loading the last checkpointed state back into the configurable IC,the process re-starts (at 2645) the user-design operation of the ICunder closer user scrutiny to identify the bug. In re-starting theoperation of the IC, the inputs that the process provides to the IC arethe inputs that the IC is suppose to receive in the clock cycle that isafter the clock cycle that was last checkpointed (i.e., after the clockcycle whose state was loaded back into the IC at 2645).

After 2645, the process transitions to 2650, where an attempt is made totry to identify and resolve the bug (i.e., the mistake) that lead to theerror. If the bug can be identified and resolved at 2650, the process(at 2655) computes a new configuration bit stream and reloads this bitstream. From 2655, the process returns to 2605 to re-start the debugoperation. On the other hand, when the bug cannot be identified andresolved at 2650, the process returns to 2645 to reload the lastcheckpointed state, or any of the checkpointed states before the lastone, and then returns to 2650, where another attempt is made to identifythe bug.

FIG. 27 illustrates a debugger 2705 that directs the debugging process2600 of FIG. 26. In some embodiments, this debugger configures the clockcontrol circuitry in the clock tree of the tile arrangement to stopafter a predetermined number of clock cycles. After the predeterminednumber of clock cycles, the debugger then performs the process 2600 byusing the controller 915. The checkpointed data is stored in a memory2710 outside of the configurable IC 2700 (i.e., in a memory that is noton the same dies as the IC) in some embodiments. When a bug isencountered, the debugger loads the last checkpointed state in the tilearrangement 1410 through the controller 915. It then resumes theconfigurable IC's operation in a more deliberate manner until the bug isidentified.

The checkpointing functionality of the configuration/debug network 1400is unique. A traditional configurable IC typically cannot implement acheckpointing functionality without altering the state of the IC. Forinstance, an existing configurable IC allows the state of theconfigurable IC from being scanned out, but in the process changes thestate of the IC. The checkpointing operation of the configuration/debugnetwork 1400 also does not require the reading out of the configurationdata to read the IC's state values. The configuration/debug networkallows the UDS values to be checkpointed without reading out theconfiguration data. Also, this network allows the checkpointing of theUDS values for only certain resources, parts or regions of the IC. Inother words, the checkpointing operations of some embodiments can beconducted to checkpoint UDS values for only a portion of the IC.

IX. Fast Configuration

Another advantage of the configuration/debug network 1400 is that itallows the configurable IC or portions of it to be configured veryquickly. This is because this network is a fully pipelined configurationnetwork. The network is fully pipelined as there are sets of synchronousstorage elements (e.g., registers) at the boundary of each horizontallyand vertically aligned tile. These sets of storage elements increase thebandwidth through this network as they allow numerous and successiveframes to traverse through the tile arrangement at the same time. Inother words, it allows one frame to be sent from the configuration/debugcontroller 915 on each clock cycle. Therefore, unlike priorconfiguration networks of prior configurable ICs, the speed of theconfiguration/debug network 1400 is independent of size of the tilearrangement.

The fast configuration rate enabled by the network 1400 has manyadvantages. One such advantage is that it allows for new configurationbit streams to be loaded from outside of the configurable IC at a fastrate. This, in turn, allows functionalities to be swapped in and out ofthe configurable IC at a fast rate.

In fact, functionalities can be swapped for some parts of theconfigurable IC while other parts of the configurable IC are performinguser-design operations. FIG. 28 illustrates an example of this fastswapping of functionalities on the fly (i.e., during the user-designoperation of the IC). Specifically, this figure illustrates aconfigurable IC 2800 that is part of a communication circuit 2850 (e.g.,a router, a hub, a switch, etc.). This IC is responsible for performinga number of communication operations, including TCP/IP communications,secure transport layer operations (SSL operations), etc.

The configurable IC 2800 has a configuration controller 915 and aconfiguration network (not shown) similar to those described above. FIG.28 conceptually illustrates that the configurable IC 2800 furtherincludes at least three sets of circuit blocks, which the configurableIC uses to perform SSL operations. The three sets include a packetgrabber 2805, a packet processor 2815, and a set ofencryptors/decryptors 2820. Even though FIG. 28 conceptually illustratessuch blocks, one of ordinary skill will realize that the configurable ICmight not perform each security operation by a set of configurableresources that fits neatly within one contiguous block. FIG. 28 simplyillustrates contiguous blocks, however, to convey the notion thatdifferent sets of resources within the tile arrangement 1410 performdifferent security operations.

The packet grabber 2805 retrieves packets from the network 2855 that thegrabber identifies as packets that need IC 2800 needs to process. Thepacket grabber stores each packet in a memory 2810 for later retrievalby the packet process 2815. In some embodiments, the memory 2810 is avolatile system memory of the communication circuit 2850.

The packet processor 2815 retrieves packets stored in the memory 2810and directs the various resources of the configurable IC to process eachretrieved packet. When a particular packet is encrypted, the packetprocessor 2815 determines whether an appropriate decryptor 2820 (i.e., aset of logic and interconnect resources that have been configured toperform the appropriate decryption algorithm) for the particularencrypted packet is already loaded on the configurable IC. If so, thepacket processor directs the identified decryptor 2820 to decrypt theparticular packet.

On the other hand, when the packet processor 2815 determines that theappropriate decryptor 2820 is not loaded on the IC, the packet processor2815 directs the configuration controller to load the appropriatedecryptor onto the configurable IC 2800 from a memory 2812 of thecircuit 2850. In some embodiments, the memory 2812 is a non-volatilememory that stores a number of different configuration bit streams fordifferent functionalities (e.g., different decryption operations) thatthe configurable IC can perform.

Loading the appropriate decryptor 2820 means loading the configurationdata that configures a set of logic and routing resources to implementthe functionality of the desired decryptor. To load such configurationdata, the configuration controller 915 has to obtain the appropriateconfiguration bit stream from the memory 2812. As it receives this bitstream, the controller 915 formulates configuration packets and routesthese packets to the appropriate configurable resources of the IC alongthe configuration/debug network 1400. Such a load might involve aswapping of decryption functionality, where a set of configurableresources that were configured to serve as one decryptor are configuredagain to serve as another decryptor.

For instance, the configurable IC 2800 might be configured to perform anRC4 decryption algorithm to decrypt secured network communications. Thepacket processor 2815 might at some point detect that another cipher,such as an AES cipher, is needed to decrypt a packet. If this othercipher is not already loaded on the configurable IC, the packetprocessor 2815 directs the configuration controller 915 to load a newbit stream that configures the IC to perform the decryption operationfor the new cipher, e.g., the AES cipher. If the configurable IC doesnot have sufficient resources for the new cipher, the configurationcontroller might swap out the RC4 cipher and load the new cipher (e.g.,the AES cipher) in its place (i.e., might load configuration data thatwould configure the set of configurable resources that were configuredto serve as RC4 decryptor to serve as the other decryptor).

The above described configuration network allows the configurationcontroller 915 to load in new configuration data for one set ofconfigurable resources while other configurable resources of the IC 2800are performing user-design operations. For instance, in the aboveexample, while the configuration controller 915 is loading theconfiguration data that configures one set of circuits to implement theAES decryption cipher, the packet grabber 2805 of the IC can continuegrabbing packets and storing them in the memory 2810.

The high speed of this network allows the controller to load the newconfiguration bit stream very quickly. The ability to load differentfunctionalities on the fly is quite advantageous in this network contextas it allows for a smaller (and hence less expensive) configurable IC tohandle a diverse traffic mix on a communication network. The IC can besmaller because it does not need to store all the functionalities (i.e.,all the configuration data) that it might need at runtime, as it canswap in functionalities (i.e., configuration bit streams) rather quicklythrough its configuration/debug network.

X. Starting and Stopping User-Design Operation of the IC

The above-described configuration network allows a configurable IC toreceive a configuration bit stream that configures the IC to implement aparticular user design. A user typically uses a set of software tools todefine the configuration data stream that configures the IC (i.e.,configures the configurable circuits of the IC) to implement the user'sparticular user design.

When implementing a particular user design, a configurable IC performsuser-design operations that allow the IC to implement the particularuser design in a circuit or device. During such user-design operations,the configurable IC (1) can receive user-design input data, which areneither configuration signals nor clocking signals, and (2) can processthese signals to implement the particular user design in a circuit ordevice. Accordingly, in some cases, a configurable IC performsuser-design operations when it receives and processes user-design inputdata and provide user-design output data. For instance, when theconfigurable IC performs user-design operations, its configurable logiccircuits in some cases can receive user-design input data, computefunctions based on the user-design input data, and output their resultsto other circuits inside or outside of the IC. In other contexts, aconfigurable IC might implement a user design that simply directs the ICto generate output without receiving any user-design input.

When a configurable IC performs user-design operations, its circuitstypically receive clocking signals that allow them to processuser-design signals. Examples of such clocking signals include (1)clocking signals applied to input/output buffer circuits that allowthese circuits to receive and output user-design data, (2) clockingsignals applied to the configurable logic circuits that allow thesecircuits to compute user-design functions, and/or (3) clocking signalsapplied to the IC's configurable interconnect circuits that allow thesecircuits to perform user-design connection operations. In case of areconfigurable IC that has reconfigurable circuits that receivedifferent configuration data sets loaded on the IC, the clock signalscan also include clock signals that allow the reconfigurable circuits tostep through the different configuration data sets.

In some embodiments, the user-design operation of the IC stops when theIC stop receiving user-design input data and/or stops providinguser-design output data. The user-design operation of the IC stops insome embodiments when the clock signals that allow the configurablecircuits to process user-design data are stopped (e.g., are maintainedat a particular level). In case of a reconfigurable IC that hasreconfigurable circuits that receive different configuration data setsloaded on the IC, stopping the clock signals can prevent thereconfigurable circuits from stepping through the differentconfiguration data sets.

While the invention has been described with reference to numerousspecific details, one of ordinary skill in the art will recognize thatthe invention can be embodied in other specific forms without departingfrom the spirit of the invention. For instance, several embodiments weredescribed above by reference to particular number of inputs, outputs,bits, and bit lines. One of ordinary skill will realize that thesevalues are different in different embodiments. Thus, one of ordinaryskill in the art would understand that the invention is not to belimited by the foregoing illustrative details, but rather is to bedefined by the appended claims.

What is claimed is:
 1. A method of accessing an integrated circuit (IC)comprising a plurality of user design state (“UDS”) circuits and aplurality of configurable circuits for configurably performing aplurality of operations, the method comprising: addressing the pluralityof the UDS circuits concurrently while the IC is performing user-designoperations; and after addressing the plurality of the UDS circuits,concurrently reading state values stored by the addressed UDS circuitsat least a user-design operating frequency while the IC is performinguser-design operations.
 2. The method of claim 1, wherein said pluralityof UDS circuits are arranged in a tile arrangement comprising an arrayof rows and columns, and wherein said concurrently reading state valuescomprises reading values from UDS circuits in a plurality of differenttiles in different rows and columns.
 3. The method of claim 1, whereinsaid concurrently reading state values stored by the addressed UDScircuits occurs in a plurality of successive clock cycles.
 4. The methodof claim 1, wherein said concurrently reading state values stored by theaddressed UDS circuits comprises transmitting at least one readinstruction.
 5. The method of claim 4 further comprising streaming UDSstate values.
 6. The method of claim 5, wherein said streaming comprisesrepeatedly outputting said state values.
 7. The method of claim 1,wherein said concurrently reading state values comprises concurrentlyreceiving state values from addressed UDS circuits into a trace buffer.8. The method of claim 1 further comprising defining merge values forspecifying how state values from two different UDS circuits should mergeonto a shared resource.
 9. The method of claim 8, wherein the sharedresource is a bus line.
 10. The method of claim 8 further comprisingoutputting the read state values and merged state values from the IC.11. An integrated circuit (IC) comprising: a plurality of configurablecircuits for configurably performing a plurality of operations; aplurality of user design state (UDS) circuits; and a networkcommunicatively coupled to the UDS circuits, said network forconcurrently addressing a plurality of the UDS circuits while the IC isin a user design operation mode.
 12. The IC of claim 11, wherein saidUDS circuits are arranged in a plurality of tiles.
 13. The IC of claim12, wherein said IC comprises at least one memory array within saidtiles.
 14. The IC of claim 11, wherein the network is arranged to writevalues concurrently to a plurality of different tiles.
 15. The IC ofclaim 11, wherein the UDS circuits are configurable interconnectcircuits that are configured as state elements.
 16. The IC of claim 11,wherein said network is a first network and wherein said plurality ofconfigurable circuits comprises a plurality of configurable logiccircuits for performing logic functions and a plurality of configurablerouting circuits that form a second network for routing user databetween said configurable logic circuits.
 17. A method of accessing anintegrated circuit (IC) comprising a plurality of configurable circuitsfor configurably performing a plurality of operations, said ICcomprising a plurality of user design state (“UDS”) circuits, the methodcomprising: addressing a plurality of the UDS circuits concurrentlywhile the IC is performing user-design operations; and after addressingthe plurality of the UDS circuits, concurrently writing state values tothe addressed UDS circuits while the IC is performing user-designoperations.
 18. The method of claim 17, wherein said addressingcomprises transmitting an address specifying a type of circuit in theIC.
 19. The method of claim 17, wherein said addressing comprisestransmitting a data packet comprising an address specifying a globaltile type and a specific tile type.
 20. The method of claim 17, whereinsaid addressing comprises transmitting a data packet comprising anaddress specifying global tile type and a tile address that identifies aplurality of tiles being addressed.
 21. The method of claim 1, whereindifferent sets of the plurality of UDS circuits are grouped in aplurality of different tiles, wherein addressing the plurality of UDScircuits concurrently comprises, for each of a plurality of tiles:transmitting a packet comprising a first frame containing a firstaddress for the tile over a network of the IC and transmitting a secondframe containing a second address for a particular UDS circuit withinthe tile.
 22. The method of claim 1, wherein the user-design operationsare performed by configurable circuits for which configuration data wasloaded onto the IC at a boot up time of the IC.